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Our  world  is  so  constructed  that  the  physical  and  material  benefits 
we  most  desire  are  sprinkled  with  the  seeds  of  disaster.  For  example, 
the  search  for  fertile  fields  often  leads  us  to  floodplains  and  our  attempt 
to  make  less  fertile  fields  productive  forces  us  to  rely,  at  some  risk, 
on  fertilizers,  pesticides,  and  fungicides.  The  wonder  drugs  that 
maintain  our  health  carry  side  effects  proportional  to  their  potency, 
and  the  comforts  and  conveniences  of  energy  are  enjoyed  at  the  risk  of 
damage  from  a host  of  pollutants.  Modern  man  has  some  control  over  the 
level  of  risk  he  faces,  but  reduction  of  risk  often  entails  reduction 
of  benefit  as  well. 

The  regulation  of  risk  poses  serious  dilemmas  for  society.  Policy 
makers  are  being  asked,  with  Increasing  frequency,  to  "weigh  the  benefits 
against  the  risks"  when  making  decisions  about  social  and  technological 
programs.  These  Individuals  often  have  highly  sophisticated  methods  at 
their  disposal  for  gathering  Information  about  problems  or  constructing 
technological  solutions.  When  It  comes  to  making  decisions,  however,  they 
typically  fall  back  upon  the  technique  which  has  been  relied  upon  since 
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antiquity  Intuition.  The  quality  of  their  Intuitions  sets  an  upper  limit 
on  the  quality  of  the  entire  declslon-maklng  process  and,  perhaps,  the 
quality  of  our  lives. 

The  purpose  of  this  paper  Is  to  explore  the  role  that  the  psychological 
study  of  decision  processes  can  play  In  facilitating  societal  risk  taking. 
Over  the  past  25  years,  empirical  and  theoretical  research  on  decision  making 
under  risk  has  produced  a body  of  knowledge  that  should  be  of  value  to 
those  who  seek  to  understand  and  improve  societal  decisions.  After  reviewing 
aspects  of  this  research  that  we  believe  particularly  relevant,  we  will 
focus  on  some  of  the  many  Important  questions  that  remain  unanswered,  even 
unstudied.  There  1s  an  urgent  need  to  link  the  study  of  man's  judgmental 
and  declslon-maklng  capabilities  to  the  making  of  decisions  that  affect 
the  health  and  safety  of  the  public.  We  are  not  suggesting  that  the  re- 
searcher abandon  the  laboratory  completely,  but  It  seems  time  for  him 
to  look  out  the  window,  and  even  foray  occasionally  Into  the  real  world. 
Fortuitously,  It  seems  likely  that,  while  commuting  between  the  real  world 
and  the  laboratory,  he  will  uncover  many  stimulating  basic  problems  lurking 
In  applied  guise. 

The  paper  is  organized  around  three  questions:  (a)  What  are  some  of 

the  basic  policy  Issues  regarding  societal  risk?  (b)  What  do  we  already 
know  that  Is  relevant?  (c)  What  more  do  we  need  to  know  and  how  might  we 
acquire  that  knowledge? 

Basic  Issues 

The  Issues  Involved  In  policy  making  for  societal  risks  can  best  be 
discussed  within  the  contexts  of  specific  problem  areas.  Two  such  areas 
are  presented  In  this  section. 
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Natural  Hazards 

Natural  hazards  constitute  an  enormous  problem.  For  example,  the  mean 
annual  cost  of  natural  disasters  In  the  U.S.  Is  approaching  $10  billion 
(Wiggins,  1974).  A major  earthquake  in  an  urban  area  could  cause  $20 
billion  In  property  damage  (Gillette  & Walsh,  1971),  not  to  mention  the 
accomoanying  human  misery,  anguish,  and  death. 

The  question  facing  public  policy  makers  Is:  What  sorts  of  measures 

should  be  employed  to  maximize  the  benefits  of  one's  natural  environment, 
while,  at  the  same  time,  minimizing  the  social  and  economic  disruption  due 
to  disasters?  In  the  case  of  floods,  the  policy  options  that  have  been 
tried  or  considered  Include  compulsory  Insurance,  flood  control  systems, 
strict  regulation  of  land  usage,  and  massive  public  relief  to  victims. 

Not  surprisingly,  technologically  ad.anced  countries  have  opted  for 
technological  solutions  such  as  dams.  It  Is  now  recognized  that,  ironically, 
fhese  well-intended  programs  have  often  exacerbated,  the 
problem.  Although  the  U.S.  government  has  spent  more  than  $10  billion 
since  1936  on  flood  control  structures,  the  mean  annual  toll  of  flood  losses 
has  risen  steadily  (White,  1964).  The  damage  Inflicted  upon  Pennsylvania  in 
1972  by  flooding  associated  with  Hurricane  Agnes  exceeded  $3  billion  despite  the 
fact  that  the  area  was  protected  by  some  66  dams.  Apparently,  the  partial 
protection  offered  by  dams  gives  residents  a false  sense  of  security  and 
promotes  overdevelopment  of  the  flood  plain.  As  a result  of  this  over- 
development, when  a rare  flood  does  exceed  the  capacity  of  the  dam, 
the  damage  Is  catastrophic.  Perpetuating  the  problem,  the  victims  of  such 
disasters  typically  return  and  rebuild  on  the  same  site  (Burton,  Kates,  & 

White,  1968).  Thus,  flood  control  dams  have  been  called  technological  man's 
ultimate  folly.  The  lesson  to  be  learned  Is  that  technological  solutions  are 
likely  to  be  inadequate  without  knowledge  of  how  they  will  affect  the  decision 
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making  of  individuals  at  risk. 

The  current  focus  of  debate  over  public  policy  is  the  question  of 
compulsory  disaster  Insurance.  Kunrcuther  (1973)  noted  that,  whereas  few 
individuals  protect  themselves  voluntarily  against  the  consequences  of  natural 
disasters,  many  turn  to  the  federal  government  after  suffering  losses. 

Federal  relief  funds  have  been  ample  enough  to  leave  many  individuals  better 
financially  after  the  disaster  than  before.  As  a result,  the  taxpayer 
is  burdened  with  financing  the  recovery  for  those  who  could  have  financed 
much  of  their  own  recovery  by  purchasing  insurance.  Kunreuther  and  others 
have  argued  that  both  property  owners  and  the  federal  government  would  be  better 
off  financially  under  an  insurance  program  than  under  a disaster  relief 
program.  They  recommend  federal  flood-insurance  policies  which  both  shift 
the  burden  of  disasters  from  the  general  taxpayer  to  individuals  living  in 
hazard-prone  areas  and  promote  wiser  decisions  regarding  use  of  flood  plains. 

One  possille  option  would  allow  individuals  to  qualify  for  federally  subsidized 
insurance  only  after  their  communities  had  taken  steps  towards  reducing  losses 
by  enforcing  land  use  measures  and  building  codes.  Another  would  make 
eligibility  for  federal  disaster  relief  loans  contingent  upon  the  individual's 
having  had  at  least  some  hazard  insurance.  A third  would  set  insurance 
rates  proportional  to  the  magnitude  of  risk  in  order  to  inform  residents  of 
those  risks  and  deter  development  of  high  risk  areas. 

Without  a better  understanding  of  how  people  perceive  and  react  to  risks, 
however,  there  is  no  way  of  knowing  whether  any  of  these  options  would  have 
the  desired  effect.  For  example,  although  there  is  evidence  that  people 
will  not  voluntarily  insure  themselves  even  if  the  rates  are  highly  subsidized, 
the  reasons  for  this  are  unknown.  Knowledge  of  the  interplay  between 
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psychological,  economic,  and  environmental  factors  as  they  determine 
insurance  purchasing  might  suggest  ways  to  increase  voluntary  purchases — 
or  indicate  the  need  for  a compulsory  Insurance  program. 

Nuclear  Power 

The  problem  of  deciding  society’s  level  of  dependence  upon  nuclear 
energy  is  so  well  known  as  to  require  little  introduction.  Policy  makers 
in  this  area  face  not  only  the  problems  of  guessing  how  the  public  will 
react  to  various  plans,  but  also  the  question  of  weighing  the  risks  and 
benefits  of  a technology  for  which  relevant  operating  experience  is  so 
limited  that  they  must  extrapolate  far  beyond  available  data. 

One  major  issue  in  the  nuclear  power  controversy  is  determining  the  locus 
of  decision-making  authority  and  the  nature  and  amount  of  public  input.  At 
one  extreme  are  those  who  argue  that  decisions  about  nuclear  development 
should  be  left  to  technical  experts  and  to  policy  makers  trained  In 
sophisticated  decision  analytic  techniques.  Resistance  to  this  view  is 
exemplified  by  Benenberg  (1974)  who  insisted  that  "Nuclear  safety  is  too 
Important  to  be  left  tc  the  experts.  It  Is  an  issue  that  should  be  resolved 
from  the  point  of  view  of  the  public  Interest,  which  requires  a broader 
perspective  than  that  of  tunnel— visioned  technicians." 

At  present,  weighing  of  benefits  vs.  risks  has  degenerated  into  a 
heated  controversy  over  the  magnitude  of  the  risks  from  loss-of-coolant 
accidents,  sabotage,  theft  of  weapon's  grade  materials,  and  long-term  storage 
of  wastes.  Some  experts  argue  that  nuclear  power  is  extraordinarily  safe; 
others  vigorously  dissent  and  have  mobilized  numerous  public  interest 
groups  in  opposition  to  the  nuclear  menace.  The  development  of  nuclear 
power  has  been  brought  to  near  standstill  bv  adverse  public  opinion. 

Ralph  Nader's  battlecry  portends  one  likely  result  of  public  participation: 
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"Let  me  make  a prediction  here.  I don't  think  that  there 
will  be  another  nuclear  plant  built  in  this  country.  . . after 
five  years.  I think  that  there  is  going  to  be  the  biggest  environ- 
mental, legal,  legislative,  executive  branch,  citizen,  consumer  battle 
in  the  history  of  the  country.  And  what  happened  to  the  SST  will  be 
a spring  picnic  compared  to  the  struggles  that  are  going  to  come 
forward  on  nuclear  fission  power"  (Nader,  1974). 

If  the  opponents  of  nuclear  power  are  right  about  the  risks,  every  reactor 

built  is  a catastrophe.  If  they  are  wrong,  following  their  advice  may  be 

equally  costly  to  society. 

What  contributions  can  cognitive  psychologists  make  towards  resolution 
of  this  controversy?  Several  possibilities  exist.  First,  they  can  help 
develop  Judgmental  techniques  to  assist  engineers  in  assessing  probabilities 
of  failure  for  systems  in  which  relevant  frequentlstlc  experience  is  lacking. 
Second,  they  can  attempt  to  clarify,  from  a psychological  standpoint,  the 
advantages  and  disadvantages  of  various  methods  of  performing  risk-benefit 
evaluations  and  determining  acceptable  levels  of  risk.  Third,  they  can 
assist  the  layman  trying  to  understand  what  the  professionals'  analyses 
mean.  Even  the  mos;  astute  technical  analysis  will  be  of  little  value  if 
its  assumptions  and  results  cannot  be  communicated  accurately  to  the 
individuals  who  bear  ultimate  decision-making  responsibility.  Fourth, 
psychological  study  of  man's  ability  to  think  rationally  about  probabilities 
and  risks  will  be  essential  in  determining  the  appropriate  roles  of  expert 
and  layman  in  the  decision-making  process. 

Psychological  Knowledge  Relevant  to  Societal  Risk  Taking 
Early  Work 

The  classic  view  of  man's  higher  mental  processes  assumes  him  to  be 
an  intellectually  gifted  creature.  A statement  typical  of  this  esteem  was 
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expressed  by  economist  Frank  Knight:  "We  are  so  built  that  what  seems 

reasonable  to  us  is  likely  to  be  confirmed  by  experience  or  we  could  not  live 
in  the  world  at  all"  (Knight,  1921.  p.  227). 

^^th  the  dawn  of  the  computer  era  and  its  concern  for  information  proces- 
sing by  man  and  machine,  a new  picture  of  man  began  to  emerge.  Miller  (1956) 
in  his  famous  study  of  classification  and  coding,  show^;d  that  there  are 


severe  limitations  on  our  ability  to  process  sensory  signals.  About  the  same 
time,  the  close  observation  of  performance  in  concept  formation  tasks  led 
Bruner,  Goodnow,  and  Austin  (1956)  to  conclude  that  the  task  Imposed 

a condition  of  "cognitive  strain"  that  their  subjects  tried  to 

reduce  by  means  of  simplification  strategies.  The  processing  of  conceptual 

Information  is  currently  viewed  as  a serial  process,  constrained  by  limited 
short-term  memory  and  a slow  storage  in  long-term  memory  (Newell  & Simon, 

1972).  With  regard  to  symbol  manipulation,  we  are  certainly  no  match  for 
a computer. 

But  what  of  the  more  direct  forms  of  decision-making  activities  we 
humans  use  to  attain  our  objectives?  Here,  too,  classic  assumptions  of 
rationality  have  been  challenged  on  psychological  grounds.  A leading  challenger 
has  been  Simon  (1957)  whose  theory  of  "bounded  rationality"  asserts  that 
the  cognitive  limitations  of  the  decision  maker  force  him  to  construct  a 
simplified  model  of  the  world  in  order  to  deal  with  it.  Simon  argued  that 
the  decision  maker 


1 ’ °®"®ves  rationally  with  respect  to  this  [simplified] 

model,  and  such  behavior  Is  not  even  approximately  optimal  with  reapect 

way  Tl  h“th'  ,■  P-b'^understaS  hT^ 

will  J almplltled  model  Is  constructed,  and  Its  construction 

Llvl^r  tM  w P^y^hPloSleal  properties  as  a per- 

g,  thinking,  and  learning  animal"  (Simon,  1957,  p.  198). 
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Recent  Studies  of  Probabilistic  Information  Processing 

Because  of  the  importance  of  probabilistic  reasoning  to  decision  making, 
a great  deal  of  recent  experimental  effort  has  been  devoted  to  understanding 
how  people  perceive  and  use  the  probabilities  of  uncertain 
events.  By  and  large,  this  research  provides  dramatic  support  for  Simon's 
concept  of  bounded  rationality.  The  experimental  results  indicate  that 
people  systematically  violate  the  principles  of  rational  decision  making 
when  judging  probabilities,  making  predictions,  or  otherwise  attempting  to 
cope  with  probabilistic  tasks.  Frequently  these  violations  can  be  traced 
to  the  use  of  judgmental  heuristics  or  simplification  strategies.  These  heuris- 
tics may  be  valid  in  some  circumstances  but  in  others  they  lead  to  biases  that 
are  large,  persistent,  and  serious  in  their  implications  for  decision  making. 
Because  much  of  this  research  has  been  summarized  elsewhere  (Slovic, 

Kunreuther  & White,  1974;  Tversky  & Kahneman,  1974),  coverage  here  will  be 
brief. 

sample  Implications.  After  questioning  a large  number  of  psycho- 
logists about  their  research  practices  and  studying  the  designs  of  experiments 
reported  in  psychological  journals,  Tversky  and  Kahneman  (1971)  concluded 
that  these  scientists  seriously  underestimated  the  amount  of 

error  and  unreliability  Inherent  in  small  samples  of  data.  They  expected 
samples  drawn  from  a given  population  to  be  more  similar  to  one 
another  and  to  the  population  than  sampling  theory  predicts.  As  a result, 
these  scientists  (a)  had  unreasonably  high  expectations  about  the  repli- 
cability of  results  from  a single  sample;  (b)  had  undue  confidence  in 
early  results  from  a few  subjects;  (c)  gambled  their  research  hypotheses  on 
small  samples  without  realizing  the  extremely  high  odds  against  detecting 
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the  effects  being  studied;  and  (d)  larely  attributed  any  unexpected  results 
to  sampling  variability  because  they  found  a causal  explanation  for  every 
disci  epancy.  Results  similar  to  these  in  quite  different  contexts  have 
been  obtained  by  Berkson,  Magath  and  Hum  (1939)  and  Brehmer  (1974). 

However,  people  are  not  always  incautious  when  drawing  inferences  from  samples 
of  data.  Under  certain  circumstances  they  become  quite  conservative, 
responding  as  though  data  are  much  less  diagnostic  than  they  truly  are 
(Edwards,  1968). 

In  a study  using  Stanford  undergraduates  as  subjects,  Kahneman 
and  Tversky  (1972)  found  that  many  of  these  subjects  did  not  understand  the 
fundamental  principle  of  sampling — that  the  validity  of  a sample  increases 
as  the  sample  size  gets  larger.  They  concluded  that  "For  anyone  who 
would  wish  to  view  man  as  a reasonable  intuitive  statistician,  such  results 
are  discouraging"  (p.  445). 

of  prediction.  Kahneman  and  Tversky  (1973)  contrasted  the  rules 
that  determined  peoples'  intuitive  predictions  with  the  normative  principles 
of  statistical  prediction.  According  to  the  normative  theory,  prior  prob- 
abilities or  base  rates,  which  summarize  what  we  knew  before  receiving 
evidence  specific  to  the  case  at  hand,  should  remain  relevant  even  after 
such  evidence  is  obtained.  People  were  found  to  behave  differently.  They 
relied  almost  exclusively  on  the  specific  information  and  neglected  the 
prior  probabilities.  Similar  results  have  been  obtained  by  Hammerton 
C1974)  and  by  Lyon  and  Slovlc  (1975). 

Another  normative  principle  is  that  the  variance  of  one's  predictions 
should  be  sensitive  to  the  validity  of  the  Informtlon  on  vhlth  the  predictions 
are  based.  It  validity  Is  not  perfect,  predictions  should  be  regressed 
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towards  some  central  value.  Furthermore,  the  lower  the  validity 
of  the  Information  on  which  predictions  are  based,  the  greater  the  regression 
should  be.  Kahneman  and  Tversky  (1973)  observed  that  otherwise  intelli- 
gent people  have  little  or  no  intuitive  understanding  of  the  concept  of 
regression.  They  fall  to  expect  regression  in  many  situations  when 
it  Is  bound  to  occur  and,  when  they  observe  it,  they  typically  Invent 
complex  but  spurious  explanations.  People  fall  to  regress  their  pre- 
dictions towards  a central  value  even  when  they  are  using  information  that 
they  themselves  consider  of  low  validity. 

Kahneman  and  Tversky  also  found  that  the  Internal  consistency  of 
a pattern  of  predictive  cues  is  a major  determiner  of  confidence  in  one's 
predictions  based  on  these  cues.  The  normative  model  asserts  that, 
given  input  variables  of  stated  validity,  accuracy  of  prediction  decreases 
as  redundancy  increases,  However,  people  tend  to  have  greater  confidence  in 
predictions  based  upon  highly  redundant  or  correlated  predictor  variables. 
Thus,  the  effect  of  redundancy  on  confidence  is  opposite  what  it  should  be. 

Availability  bias.  Another  form  of  judgmental  bias  occurs  from  the 
use  of  the  "availability"  heuristic,  (Tversky  and  Kahneman,  1973).  This 
heuristic  involves  judging  the  probability  or  frequency  of  an  event  by  the 
ease  with  which  relevant  Instances  are  imagined  or  by  the  number  of  such 
Instances  that  are  readily  retrieved  from  memory.  In  life,  instances  of 

frequent  events  are  typically  easier  to  recall  than  instances  of  less  frequent 
events  and  likely  occurrences  are  usually  easier  to  imagine  than  unlikely  ones 
thus  mental  availability  will  often  be  a valid  cue  for  the  assessment  of 
frequency  and  probability.  However,  availability  is  also  affected  by  many 
subtle  factors  which  are  unrelated  to  actual  frequency,  such  as  recency  and 
emotional  sal lency.  If  the  availability  heuristic  is  applied,  then  factors 
that  increase  the  availability  of  Instances  should  correspondingly  Increase 
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the  perceived  frequency  and  subjective  probability  of  the  events  under 
consideration.  Thus,  reliance  on  availability  may  result  in  serious  and 

predictable  biases  in  judgment  as  the  experiments  by  Tversky  and  Kahneman 
demonstrate. 

--oring  biases.  Bias  also  occurs  when  a judge  attempts  to  ease 
the  strain  of  processing  information  by  following  a process  of  anchoring 
and  adjustment.  In  this  process,  a natural  starting  point  is  used  as 
a first  approximation  to  the  judgment,  an  anchor,  so  to  speak.  This 
anchor  is  then  adjusted  to  accommodate  the  Implications  of  the  additional 
information.  Typically,  the  adjustment  is  a crude  and  imprecise  one 
which  falls  to  do  justice  to  the  Importance  of  additional  information. 

Recent  work  by  Tversky  and  Kahneman  (1974)  demonstrates  the 
fact  that  adjustments  tend  to  be  insufficient.  They  asked  .subjects  questions 
such  as  "What  is  the  percentage  of  people  in  the  U.  S.  today  who  are  age 
55  or  older?"  They  gave  the  .subjects  starting  percentages  that  were 
randomly  chosen  and  asked  the  subjects  to  adjust  these  starting  points 
until  they  reached  their  best  estimate.  Because  of  insufficient  adjustment, 
those  whose  starting  points  were  too  high  ended  up  with  higher  estimates 
than  those  who  started  with  a value  that  was  too  low. 

Application  of  the  anchoring  and  adjustment  heuristic  is  also  hypothe- 
sized to  produce  the  serious  bias  that  occurs  when  people  attempt  to  express 
the  degree  to  which  they  are  uncertain  about  an  estimate  or  prediction. 
Specifically,  in  studies  by  Alpert  and  Raiffa  (1968)  and  .Schaefer  and 
Borcherding  (1973),  subjects  were  given  almanac  questions  such  as  the  following 

How  many  foreign  cars  were  imported  into  the  U.S.  in  1968? 

a)  Make  a high  estimate  such  that 


you  feel  there  is  only  a IZ 
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probability  the  true  answer  would  exceed  your  estimate. 

b)  Make  a low  estimate  such  that  you  feel  there  is  only  a 1% 
probability  the  true  answer  would  be  below  this  estimate. 

If  subjects'  answers  were  unbiased,  we  should  expect  the  true  answers  to 
such  questions  to  fall  between  the  upper  and  lower  bounds  98%  of  the  time. 

What  is  typically  found,  however,  is  that  subjects  set  the  bounds  much 
too  narrowly.  The  true  value  exceeds  the  bounds  as  much  as  <^>0%  of  the  time 
across  many  subjects  answering  many  questions.  Subjects  appear  to  use 
a computational  algorithm  to  devise  a best  guess  which  serves  as  an 
anchor.  Their  boundary  estimates  are  then  determined  by  adjusting  this 
best  estimate.  These  adjustments  tend  to  be  insufficient  in  magnitude, 
falling  to  do  justice  to  the  many  ways  in  which  the  initial  estimate  can  be 
in  error. 

Lichtenstein  and  Slovic  (1971,  1973)  observed  that  anchoring  and  adjustment 
leads  people  to  make  judgments  about  the  attractiveness  of  gambles  that  are 
inconsistent  with  their  choices  among  those  same  gambles.  When 

confronted  with  a pair  of  gambles,  one  offering  a high  probability  to  win  a modest 
amount  and  the  other  offering  a modest  probability  to  win  a large  amount,  subjects 
tend  to  choose  the  high  probability  gamble  as  most  attractive  but  state  higher 
selling  or  buying  prices  for  the  high  payoff  gambles.  This  inconsistency 
derives  from  the  fact  that  people  who  find  a gamble  attractive  use  the  amount  to 
win  as  a natural  starting  point  for  attaching  a monetary  price  to  it.  l\fhen  they 
adjust  the  amount  to  win  downward,  to  take  probability  and  amount  to  lose 
into  account,  the  adjustment  tends  to  be  rather  small,  leaving  their  final 
price  rather  close  to  the  amount  to  win.  Thus  gambles  with  large 

winning  payoffs  are  overvalued  by  this  monetary  index  of  attractiveness. 
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HlndBlRht  biases.  A series  of  experiments  by  Fiachhoff  (1974,  1975;  . 

Fischhoff  & Beyth,  1975)  has  examined  the  phenomenon  of  hindsight.  Fischhoff 
found  that  being  told  some  event  has  happened  increases  our  feeling  that  it 
was  inevitable.  We  are  unaware,  however,  of  this  effect  of  knowing  how  the  outcome 
turned  out,  and  tend  to  believe  that  this  inevitability  was  apparent  in 
foresight,  before  we  knew  what  happened.  In  retrospect,  we  tend  to  believe 
that  we  (and  others)  had  a much  better  idea  of  what  was  going  to  happen 
than  we  actually  did  have.  Fischhoff  (1974)  shows  how  such  misperceptions 
can  seriously  prejudice  the  evaluation  of  decisions  made  in  the  past  and 
limit  what  is  learned  from  experience. 


Comment 

Since  these  experimental  results  greatly  contradict  our  traditional 
image  of  man,  it  is  reasonable  to  question  whether  the  observed  inadequacies 
of  probabilistic  thinking  would  persist  outside  the  laboratory  in  situations 
where  decision  makers  use  familiar  sources  of  information  to  make  decisions 
that  are  personally  Important  to  them. 

Much  evidence  suggests  that  the  laboratory  results  will  generalize. 
These  cognitive  limitations  appear  to  pervade  a wide  variety  of  tasks 
where  intelligent  individuals  served  as  decision  makers,  often  under 
conditions  that  maximized  motivation  and  involvement.  For  example,  the 
subjects  studied  by  Tversky  and  Kahneman  (1971)  were  scientists,  highly 
trained  in  statistics,  evaluating  problems  similar  to  those  they  faced  in 
theli  own  research.  Qverdependence  on  specific  evidence  and 
neglect  of  base  rates  has  been  observed  among  psychometricians  responsible 
for  the  development  and  use  of  psychological  tests  (Meehl  & Rosen,  1955). 
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When  Lichtenstein  and  Slovic  (1971)  observed  anchoring  bias  in  subjects' 
evaluations  of  gambles,  they  repeated  the  study,  with  identical  results, 
on  the  floor  of  a Las  Vegas  casino  (Lichtenstein  & Slovic,  1973). 

Particularly  relevant  to  the  present  paper  is  evidence  illustrating 
these  sorts  of  biases  in  individuals  attempting  to  cope  with  natural 
disasters.  For  example,  availability  biases  are  apparent  in  the  behavio- 

of  residents  on  the  flood  plain.  Kates  (1962,  p.  140)  writes: 

"A  major  limitation  to  human  ability  to  use  improved  flood  hazard 
information  is  a basic  reliance  on  experience.  Men  on  flood  plains 
appear  very  much  to  be  prisoners  of  their  experience.  . . Recently 
experienced  floods  appear  to  set  an  upward  bound  to  the  size  of  loss 
with  which  managers  believe  they  ought  to  be  concerned." 

Kates  further  attributes  much  of  the  difficulty  in  achieving  better 

flood  control  to  the  inability  of  individuals  to  conceptualize  floods 

that  have  never  occurred"  (p.  88).  He  observes  that,  in  making  forecasts 

of  future  flood  potential,  individuals  "are  strongly  conditioned  by  their 

immediate  past  and  limit  their  extrapolation  to  simplified  constructs, 

seeing  the  future  as  a mirror  of  that  past"  (p.  88).  A more  detailed 

linkage  between  psychological  research,  bounded  rationality,  and  behavior 

in  the  face  of  natural  hazards  is  provided  by  Slovic,  Kunreuther,  and  White 
(1974). 

One  additional  implication  of  the  research  on  man's  limited  ability  to 
process  probabilistic  Information  deserves  comment.  Most  of  the  discussions 
of  cognitive  strain"  and  "limited  capacity"  that  are  derived  from  the  study  of 
problem  solving  and  concept  formation  depict  man  as  a computer  which  h.as 
the  right  programs  but  cannot  execute  them  properly  because  its  central 
processor  is  too  small.  The  biases  due  to  availability  and  anchoring  certainly 


15 


sre  congruent  with  this  analogy.  But  the  mlsj udgment  of  sampling  variability 
and  the  errors  of  prediction  Illustrate  more  serious  def Icle'icles.  Here 
we  see  that  man  s judgments  of  Important  probabilistic  phenomena  are  not 
merely  biased  but  are  fundamentally  wrong . Returning  to  the  computer 
analogy,  It  appears  that  man  lacks  the  correct  programs  for  many  Important 
Judgmental  tasks. 

How  could  It  be  that  we  lack  adequate  programs  for  probabilistic 
thinking?  Slnshelmer  (1971)  argues  that  man's  brain  has  been  designed 

by  evolution  to  cope  with  certain  very  real  problems  In  the  Immediate, 
external  world  and  thus  lacks  the  conceptual  framework  with  which  to  encom- 
pass many  phenomena.  He  comments  on  how  much  more  difficult  It  Is  to  teach  a 
17-year-old  a few  laws  of  physics  than  to  teach  him  ^er  to  drive  a car: 

To  drive  a car,  a 17-year-old  makes  use  of  a set  of  routines  long 
since  programmed  Into  the  primate  brain.  To  gauge  the  speed  of  an 
approaching  car  and  maneuver  accordingly  Is  not  that  different  from 
[gauging]  the  speed  of  an  approaching  branch  and  [reacting]  accord- 
ingly as  one  swings  through  the  trees  ....  Whereas  to  solve  a 
problem  In  diffraction  imposes  an  intricate  and  entirely  unfamiliar 
task  upon  a set  of  neurons"  (Slnshelmer,  1971,  p.  21). 

In  the  past  as  In  the  present,  man's  decision  making  has  largely 
ignored  uncertainty,  relying  either  on  habit  (tradition)  or  simple  deter- 
ministic rules.  Following  Slnshelmer' s reasoning,  it  might  be  argued  that 
man  has  not  had  the  opportunity  to  evolve  an  intellect  capable  of  dealing 
conceptually  with  uncertainty.  He  is  essentially  a trlal-and-error  learner 
and  it  remains  to  be  seen  whether  he  can  change  his  ways  In  the  nuclear  age 
when  errors  may  be  catastrophic. 

Where  Do  We  Go  From  Here? 

Suppose  that  individuals  and  society  are  unfit  to  think  rationally  about 
uncertainty.  What  are  they  to  do?  A traditional  solution  has  been  to  defer 
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to  technical  experts,  trained  in  dealing  with  uncertainty.  To  some  extent, 
this  strategy  may  work.  Adopting  it,  however,  incurs  a societal  risk  of 
quite  another  sort,  the  creation  of  a technocratic  state.  In  addition,  as 
both  the  evidence  described  above  suggests  and  the  consumer  advocates  loudly 
claim,  experts,  too.  are  prone  to  biases.  Experts  certainly  are  better  than 
lay  people  at  many  tasks  for  which  solution  algorithms  are  available,  such  as 
building  a dam  or  managing  an  inventory.  When,  however,  they  are  confronted  with 
a new  or  poorly  defined  problem  for  which  their  algorithms  fail  to  apply, 

or  for  which  they  must  supply  subjective  inputs  to  those  algorithms,  they 
may  do  no  better  than  the  rest  of  us. 

Psychological  research  can  contribute  by  identifying  the  major  sources 
of  error  in  societal  risk-taking  decisions  and  by  devising  techniques  to 
minimize  those  errors.  In  the  remainder  of  this  paper  we  shall  speculate 
about  some  of  the  directions  this  research  could  take. 

Evaluating  Low-Probability,  High— Consequence  Events 

The  most  important  public  hazards  are  events  with  extremely  low  probabi- 
lities and  extremely  great  consequences.  For  example.  Holmes  (1961)  found 
that  50%  of  the  damage  due  to  major  floods  was  caused  by  floods  whose  probabi- 
lity of  occurrence  in  any  year  was  less  than  .01.  The  city  of 
Skopje,  Yugoslavia  was  leveled  by  earthquakes  in  the  years  518,  1555,  and 
1963,  and  the  mudflow  that  took  25,000  lives  in  Yungay,  Peru,  had  swept  across 
the  same  valley  between  1,000  and  10,000  years  before.  The  probability 
of  serious  radiation  release  from  a nuclear  power  reactor  has  been  estimated 
at  between  10  ^ and  10  per  reactor  year.  Despite  the  obvious  significance 
of  understanding  how  (and  how  well)  experts  and  laymen  estimate  probabilities 
for  such  events,  there  har  been  little  or  no  systematic  study  of  this  problem 
(for  an  Important  exception  to  this  statement,  see  Selvidge,  1975).  Some 
approaches  to  this  research  are  illustrated  below. 
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^ailablllty  biases.  We  have  recently  been  studying  people's  per- 
ceptions of  low-probability,  high-consequence  events.  Our  stimuli  were  A1 
causes  of  death,  including  diseases,  accidents,  homicide,  suicide,  and  natural 
hazards.  The  probability  that  a randomly  selected  U.S.  resident  would 
succumb  to  one  of  these  events  in  a year  ranges  from  about  1 x 10~^  (botulism) 
to  1,6  X 10  (cancel^  and  8.5  x 10  (heart  disease).  We  constructed  106 
pairs  of  these  events  and  asked  a large  sample  of  college  stude.its  to  indicate, 
for  each  pair,  the  more  likely  cause  of  death  and  the  ratio  of  the  greater 
frequency  to  the  lesser  frequency. 

We  found  that  (a)  our  subjects  had  a consistent  subjective  ordering 
of  relative  frequency  for  causes  of  death;  (b)  this  subjective  ordering  often 
deviated  markedly  from  the  true  ordering;  (c)  with  a few  notable  exceptions, 

the  subjects  could  accurately  discriminate  the  pairwise  relative  frequencies 
of  causes  of  death  when  the  true  ratio  cf  greater  to  lesser  frequency  was  5:1 
or  more.  At  true  ratios  of  2:1  or  below,  discrimination  was  poor,  A subset 
of  the  detailed  results  is  presented  in  Table  1. 

Insert  Table  1 about  here 

Kates'  (1962)  research  suggests  that  evaluations  of  low-probability,  high- 
consequence  events  are  likely  to  be  influenced  greatly  by  factors  such  as 
personal  experience,  imaginability,  and  memorability.  To  determine  the 
extent  that  the  misperceptions  we  observ  d can  be  accounted  for  by  "availa- 
bflfty  factors  , we  had  a separate  group  of  subjects  rate  each  event  in 
terms  of  their  exposure  to  it  via  the  media  and  via  their  own  personal  ex- 
periences with  the  event  as  a cause  of  suffering  and  death.  From  these 
ratings,  an  availability  score  was  given  to  each  event,  a high  score  indicating 
a high  degree  of  exposure  to  the  event,  across  subjects.  The  difference  in 
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TABLE  1 


Judgments  of  Relative  Frequency  for  Selected  Pairs  of  Lethal  Events 


Less  Likely 

More  Likely 

True 

Ratio 

% Correct 
Discrimination 

Geometric  ^ 
Mean  of 
Judged  Ratios 

Asthma 

Firearm  Accident 

1.20 

80 

11.00 

Breast  Cancer 

Diabetes 

1.25 

23 

.13 

Lung  Cancer 

Stomach  Cancer 

1.25 

25 

.31 

Leukemia 

Einphysema 

1.49 

47 

.58 

Srroke 

All  Cancer 

1.57 

83 

21.00 

All  Accidents 

Stroke 

1.85 

20 

.04 

Pregnancy 

Appendicitis 

2.00 

17 

-10 

Tuberculosis 

Fire  & Flames 

2.00 

81 

10.50 

Emphysema 

All  Accidents 

5.19 

88 

269.00 

Polio 

Tornado 

5.30 

71 

4.26 

Drowning 

Suicide 

9.60 

70 

5.50 

All  Accidents 

All  Diseases 

15.50 

57 

1.62 

Diabetes 

Heart  Disease 

18.90 

97 

127.00 

Tornado 

Asthma 

20.90 

42 

.36 

Syphilis 

Homicide 

46.00 

86 

31.70 

Botulism 

Lightning 

52.00 

37 

.30 

Flood 

Homicide 

92.00 

91 

81.70 

Syphilis 

Diabetes 

95.00 

64 

2.36 

Botulism 

Asthma 

920.00 

59 

1.50 

Excess  Cold 

All  Cancer 

982.00 

95 

1490.00 

Botulism 

Emphysema  10 

,600.00 

86 

24.00 

2 

Geometric  means  less  than  1.00  Indicate  that  the  mean  ratio  was  higher  for  the  less 
likely  event.  A geometric  mean  of  .20  implies  the  mean  was  5:1  in  the  wrong  direction. 
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these  scores  for  paired  events  was  found  to  correlate  .57  with  a "mispercep- 
tion" index  for  each  pair.  Consideration  of  the  specific  events  that  appear  most 
overestimated  (botulism,  tornadoes,  accidents)  with  those  most  under- 
estimated (asthma,  emphysema,  diabetes)  further  strengthens  our  belief 

regarding  the  Importance  of  differential  media  publicity  and  Imaglnablllty 
In  probability  assessment.  Further  indirect  support  for  the  avallabllltv 

hypothesis  is  that  accuracy  Increases  somewhat  with  increasing  absolute 
frequency  (and  thus  exposure)  for  the  events  in  question.  At  present,  we 
are  studying  whether  subjects  Informed  about  the  nature  of  availability 
bias  will  perform  better  than  the  subjects  in  our  first  experiment. 

Many  other  questions  remain  to  be  asked.  For  example,  will  the  results 
of  the  first  study  vary  across  subject  populations?  Perhaps  older  people 
will  have  different  perceptions  of  the  risks  for  events  to  which  they  are 
most  susceptible.  Do  men  and  women  differ  in  their  judgments?  Are  people 
more  accurate  in  gauging  higher-probability  chronic  hazards  such  as  drowning 
or  automobile  accidents  than  rarer,  but  more  spectacular  catastrophes  such 
as  floods  and  tornadoes?  Economists  such  as  Bergstrom  (1974)  have  hypothesized 
that  there  will  be  greater  divergence  of  opinion  about  the  probability  of 
catastrophes  than  aVout  the  probability  of  death  from  chronic  sources  and 
that  this,  in  turn,  will  lead  to  more  disagreement  on  how  much  it  is  worth 

to  attempt  to  prevent  a catastrophe.  These  hypotheses  can  and  should  be  put 
to  empirical  test. 

The  important  Implication  of  this  exploratory  study  is  that,  contrary 
to  the  assumptions  of  many  policy  makers,  intelligent  individuals  may  not 
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have  valid  perceptions  about  hazardous  events  to  which  they  are  exposed. 

■E.^u^lb~bree  analysis.  The  previous  section  Illustrates  the  manner  In 

which  the  psychological  construct  of  availability  might  Influence  the  Intuitive 

assessment  of  low-probablllty,  hlgh-consequence  events.  The  present  section 

considers  the  manner  In  which  psychological  analysis,  again  using  availability 

along  with  other  considerations,  might  help  technical  experts  In  their  use 
of  sophisticated  techniques  such  as  fault-tree  analysis  to  assess  the 

probabilities  of  rare  hazards. 

When  frequentlstlc  data  for  failure  rates  of  a complex  system  are 
unavailable,  many  technical  experts  believe  that  an  analytic  estimate  can 
be  obtained  by  constructing  a fault  tree.  Construction  begins  by  listing 
all  Important  pathways  to  failure,  then  listing  all  possible  pathways  to 
these  pathways,  and  so  on.  When  the  desired  degree  of  detail  Is  obtained, 
probabilities  are  assigned  to  each  of  the  component  pathways — and  then 
combined  to  provide  an  overall  failure  rate.  For  example,  major  pathways 
in  a fault  tree  designed  to  calculate  the  probability  of  a car  failing 
to  start  would  Include  defects  in  the  battery,  starting  system,  fuel 
system.  Ignition  system,  etc.  Battery  deficiency  could,  then,  be  traced  to 
loose  terminals  or  weak  battery  charge.  The  latter  could  be  further  analyzed 
into  its  component  causes,  such  as  lights  left  on,  cold  weather,  defective 
generator,  etc.  The  likelihoods  of  these  separate  events  would  be 
combined  to  produce  an  estimate  of  the  overall  probability  of  starting 
failure. 

The  Importance  of  fault-tree  analysis  Is  demonstrated  by  Its  role  as 
the  primary  methodological  tool  in  a recently  completed  study  assessing 
the  probability  of  a catastrophic  loss  of  coolant  accident  in  a nuclear 
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power  reactor  (Rasmussen,  1974).  The  study,  sponsored  by  the  Atomic 

Energy  Commission  at  a cost  of  $2  million,  concluded  that  the  likelihood 

of  such  an  accident  ranged  between  lO’^  (for  an  accident  causing  10  deaths) 

-9 

to  10  (for  a 1,000-death  accident)  per  reactor  - year . Fault-tree 
analysis  has,  however,  recently  come  under  attack  from  -^rltlcs  who  question 
whether  It  Is  valid  enough  to  be  used  as  a basis  for  decisions  of  great 
consequence  (e.g, , Bryan,  1974). 

Psychologists  may  be  able  to  Improve  the  effectiveness  of  fault  trees 
by  Identifying  biases  which  may  afflict  fault  tree  users  and  by  shoring  up 
the  methodology.  One  methodological  problem  which  psychologists  surely 
could  address  ig  deciding  by  what  technique  (e.g.,  direct  estimation, 
paired  comparisons,  Delphi  methods)  failure  rates  for  component  parts 
should  be  estimated.  One  possible  source  of  bias  worth  Investigating  arises 
from  the  fact  that  one  rarely  has  complete  failure  rates  on  all  the  component 
parts  of  a complex  system.  Such  rates  are  typically  estimated  from  slightly 
different  parts  or  parts  that  were  developed  for  a different  purpose. 

Anchoring  and  adjustment  may  well  play  a role  here,  possibly  leading  to 
estimates  more  suitable  for  the  original  part  or  original  context  than  for 
the  one  In  question. 

Another  possible  bias  would  be  the  omission  of  relevant  pathways  to  failure 
or  disaster.  A tree  used  to  estimate  starting  failure  In  an  automobile  could,  for 
example,  be  seriously  deficient  if  It  failed  to  Include  problems  with  the  seat  belt 
system  (for  1974  models),  theft  of  vital  parts,  or  other  vandalism. 

The  dangers  of  omitting  relevant  pathways  to  disaster  should  not  be  under- 
estimated. Tbe  cartoon  by  Mauldin  dramatizes  this  problem,  reflecting  the 
recent  reports  that  the  ozone  layer,  which  protects  the  earth  from  solar 
radiation,  may  be  damaged  by  the  fluorocarbons  released  by  aerosol  products. 

In  the  Innumerable  scenarios  which  have  been  created  to  evaluate  the  "'sjor  risks 
of  technology  to  mankind,  who  would  have  thought  prior  to  this  discovery 
to  Include  hair  sprays  and  deodorants  as  lethal  agents? 
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So  THAT’S  the  one  most  likely  lo  get  us’ 
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Insert  Cartoon  about  here 

We  suspect  that,  In  general,  experts  will  not  be  adequately  sensitive 

to  those  avenues  to  disaster  that,  due  to  lack  of  knowledge,  forgetting,  or 
lack  of  imagination,  they  have  failed  to  consider.  We  would  assume  the 

expert  to  be  unaware  of  his  omissions  and,  therefore,  to  be  unduly  confident 

in  the  completeness  of  his  analysis  and  the  validity  of  his  estimate.  Experts 

who  are  unaware  of  their  own  omissions  are  likely  to  seriously  underestimate 

the  true  failure  rate.  This  hypothesis  can  surely  be  tested  experimentally. 

Even  if  technical  experts  can  be  helped  to  produce  better  estimates, 
problems  with  the  fault  tree  would  not  be  over.  With  most  societal  decisions, 
ultimate  decision-making  responsibility  lies  with  either  the  general 
public  or  political  policy  makers.  The  finest  analysis  will  be  of  little 
value  if  it  cannot  be  communicated  to  these  people.  Considerations  of  avalla- 
blllty  suggest  that  fault— tree  analysis  is  a technique  whose  results  are 
particularly  prone  to  creating  misconceptions.  For  example,  naive  observers 
of  a fault  tree  may  be  startled  by  the  variety  of  possible  pathways  to 
disaster,  some  of  which  will  be  new  and  surprising  to  them. 

Unless  they  combat  the  increased  imaginablllty  of  disaster 
pathways  by  properly  discounting  the  less  probable  paths,  they  are 
likely  to  overreact,  perceiving  the  risk  to  be  greater  than 

it  is.  Furthermore,  the  larger  and  bushier  a tree  is — in  the  detail 
with  which  specific  components  of  each  major  pathway  are  presented — the 
greater  the  misperception  may  be.  Thus,  analyses  intended  to  clarify  decision 
makers'  perceptions  may,  instead,  distort  them. 

Critics  of  nuclear  power  often  appear  to  be  playing  on  these  proclivities. 
Consider  this  message  from  Alfven  (1972):  "Fission  energy  is  safe  only 

if  a number  of  critical  devices  work  as  they  should,  if  a number  of  people 
in  key  positions  all  follow  their  instructions,  if  there  is  no  sabotage,  no 
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hijacking  of  the  transports,  ...  No  acts  of  Cod  can  be  permitted"  (p.  6). 

Although  Alfven's  statement  is  an  extreme  position,  availability  effects 
may  make  it  difficult  even  to  engage  in  unbiased  attempts  at  discussing  low 
probability  hazards  without,  at  the  same  time,  increasing  the  perceived 
probability  of  those  hazards.  This  may  explain,  in  part,  why  continued 
discussions  of  nuclear  power  risks  have  led  to  increased  resistance  to 
this  technology.  Ultimately,  public  acceptance  of  new,  high  risk  technologies 
may  be  determined  more  by  psychological  considerations  than  by  the  opinions 
of  technical  experts. 

Coherence  and  the  judged  probability  of  scenarios.  Forecasts  and 
predictions  of  high  consequence  events  are  often  developed  within  the  context 
of  scenarios.  Some  recent  examples  are  "The  Day  They  Blew  Up  San  Onofre" 
(Schlelmer,  1974),  describing  the  sabotage  of  a nuclear  reactor  and  its 
consequences,  and  "The  Oil  War  of  1976"  (Erdmann,  1975)  describing  how  the 
world  as  we  know  it  comes  to  an  end  when  the  Shah  of  Iran  decides  to  take  it 
over  with  Western  arms.  A scenario  consists  of  a series  of  events  linked 
together  in  a narrative  form.  Normatively,  the  probability  that  a multi- 
event  scenario  will  happen  is  a multiplicative  function  of  the  probabilities 
of  the  individual  links.  The  more  links  there  are  in  the  scenario,  the 
lower  the  probability  that  the  entire  scenario  will  occur.  The  probability 
of  the  weakest  link  sets  an  upper  limit  on  the  probability  of  the  entire 
narrative. 

Human  judges  don't  appear  to  evaluate  scenarios  logically.  We  have 
begun  collecting  data  that  suggest  that  the  probability  of  a multi-link 
scenario  is  judged  according  to  its  overall  coherence,  where  coherence 
represents  a global  impression  based  on  the  average  likelihood  of  all 
linkages.  Subsequent  strong  links  appear  to  "even  out"  or  compensate 
earlier  weak  links,  making  it  possible  to  construct  scenarios  whose  perceived 
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probability  Increases  as  they  become  longer,  more  detailed,  and  normatlvely 
.probable.  Consider  the  following  example  of  such  a scenario; 

"Tom  is  of  high  intelligence,  although  lacking  In  true  creativity. 

He  has  a need  for  order  and  clarity,  and  for  neat  and  tidy  systems  In 

which  every  detail  finds  its  appropriate  place.  His  writing  is  rather 
dull  and  mechanical,  occasionally  enlivened  by  somewhat  corny  puns 

and  by  flashes  of  Imagination  of  the  sci-fi  tvpe.  He  has  a strong 

drive  for  competence.  He  seems  to  have  little  feel  and  little  sympathy 

for  other  people  and  does  not  enjoy  Interacting  with  others. 

In  the  light  of  these  data,  what  is  the  probability  that  (a)  Tom  W. 
will  select  Journalism  as  his  college  major  (b)  but  quickly  become  un- 
happy with  his  choice  and  (c)  switch  to  Engineering?" 

When  subjects  were  given  the  initial  conditions  contained  in  the 
first  paragraph  and  asked  to  estimate  the  probability  of  subsequent  event 
a_,  Tom's  selection  of  journalism  as  his  college  major,  their  mean  estimate 
was  .21.  When  they  were  asked  to  estimate  the  compound  probability  o^ 
statements  a and  _b,  given  the  same  initial  conditions,  the  mean  probability 
rose  to  .39.  When  they  were  asked  to  estimate  the  compound  event  consisting 
of  statements  _b  and  £,  the  mean  probability  rose  to  .41. 

These  startling  results  suggest  that  scenarios  are  really  quite  difficult 
to  evaluate  properly.  Specifically,  well  constructed  scenarios,  ones  which 
tell  a "good  story",  may  be  accorded  much  more  credibility  than  they  deserve. 

Experiments  are  needed  to  clarify  the  cognitive  processes  used  In  the 
evaluation  of  scenarios  and  the  biases  to  which  these  processes  are  susceptible. 
It  is  Important  to  determine  the  qualities  of  scenarios  that  lead  them  to 
appear  feasible  or  not  feasible.  We  suspect  that  scenarios  mislead  by 
burying  weak  links  in  masses  of  coherent,  appealing  details  and  by  giving 
no  hint  of  alternative  scenarios  or  no  indication  of  how  the  proposed  scenario 
could  go  wrong.  Writers  of  scenarios  may  be  deceiving  themselves  as  well 
as  others  by  falling  to  Incorporate  probabilistic  considerations  Into  their 


narratives. 
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Debiaslnf>  Procedures 

Much  of  the  research  described  above  focuses  on  Identifying  Judgmental 
biases  and  understanding  their  cognitive  underpinnings.  Once  disseminated 
to  Individuals  In  non-technlcal  form,  this  knowledge  can  alert  them  to 
biases  In  their  own  judgments  as  well  as  In  the  Information  and  recommenda- 
tions they  receive  from  others.  Full  utilization  of  this  knowledge,  however,  re- 
quires the  development  of  specific  procedures  to  circumvent  and/or  correct  biases. 

Work  on  rormulatlng  and  testing  debiasing  procedures  needs  to  be  under- 
taken, and  will  depend,  of  course,  upon  further  developments  In  our  under- 
standing of  the  biases  themselves.  An  obvious  first  step  Is  simply  to 
educate  or  warn  the  judge  about  the  bias.  If  this  falls,  more  sophisticated 
techniques  will  have  to  be  devised.  For  example,  to  combat  coherence 
biases  In  the  evaluation  of  scenarios,  It  may  be  necessary  to  decompose  the 
scenario  into  Its  component  events,  estimate  conditional  probabilities 
for  Individual  events  given  preceding  developments,  and  then  combine 
these  conditional  probabilities  mathematically  to  produce  an  overall 
evaluation  (see  Edwards  & Phillips,  1964,  for  details  of  a similar  approach 
to  combat  a different  bias).  Alternatively,  one  could  Insist  on  the  pro- 
duction of  multiple  alternative  scenarios  on  any  given  topic  and  use  an  ad- 
versary approach  to  evaluation  In  which  the  merits  and  disadvantages  of  each 
are  debated.  The  whole  area  of  deblaslng  Is  open  for  development  and  in 
need  of  creative  research. 

Psychological  Considerations  In  Risk-Benefit  Analysis 

The  application  of  scientific  methods  and  formal  analyses  to  problems 
of  decision  making  originated  during  World  War  II  from  the  need  to  solve 
strategic  and  tactical  problems  In  situations  where  experience  was  either 
costly  or  Impossible  to  acquire.  One  of  the  offshoots  of  this  work  Is 
the  technique  called  "cost-benefit  analysis",  which  attempts  to  quantify  the 
prospective  gains  and  losses  from  some  proposed  action,  usually  In  monetary  terms. 
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If  the  calculated  gain  from  an  act  or  project  is  positive,  it  is 

said  that  the  benefits  outweigh  the  costs  and  its  acceptance  is  recommended, 
providing  no  other  alternative  affords  a better  cost-benefit  ratio.  A good 
example  of  this  is  the  analysis  of  auto-safety  features  by  Lave  and  Weber  (1970). 
Risk-benefit  analysis  is  a special  case  in  which  explicit  attention  is 
given  to  quantifying  costs  due  to  loss  of  life  or  limb,  pain,  and 
anguish. 

Risk-benefit  analysis,  still  in  its  early  stages  of  development,  is 
being  counted  on  to  provide  the  basic  methodological  tools  for  societal 
risk-taking  decisions.  This  nascent  methodology  clearly  has  important 
political,  economic,  social,  and  technical  components.  In  this  section, 
we  shall  explore  the  manner  in  which  certain  of  these  components  Interact 

with  various  psychological  considerations  as  risks  and  benefits  are  assessed 
and  evaluated. 

How  safe  is  safe  enough?  Any  risk-benefit  analysis  must  ultimately 
answer  the  question  "How  safe  is  safe  enough?"  Starr  (1969)  has  pro 
posed  a quantitative  technique  for  answering  this  question  based  on  the 
assumption  that  society  learns  by  trial,  error,  and  subsequent  corrective 
actions  to  arrive  at  a reasonably  optimal  balance  between  the  risks  and 
benefits  associated  with  any  activity.  This  leads  to  the  use  of  historical 
accident  and  fatality  records  to  reveal  patterns  of  "acceptable"  risk-benefit 
ratios.  Acceptable  risk  for  a new  technology  becomes  that  level  of  safety 

associated  with  ongoing  activities  having  similar  benefit  to  society. 

Starr  illustrates  his  technique  by  examining  the  relationship  between  risk 

and  benefit  across  a number  of  common  activities.  His  measure  of  risk  for 
these  hazardous  activities  is  the  statistical  expectation  of  fatalities 
per  hour  of  exposure  to  the  activity  under  consideration.  Regarding 
benefits,  Starr  distinguishes  between  voluntary  risk-taking  activities  (e.g.. 
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sports,  smokinp,,  flyinR  in  private  aircraft),  which  the  individual  can 
engage  in  or  not  according  to  his  own  value  system,  and  involuntary  activities 
Ce.g.,  commercial  aviation,  energy  production  systems)  . For  voluntary  ac- 
tivities, benefit  is  assumed  to  be  approximately  equal  to  the  average  amount 
of  money  spent  on  an  activity  by  an  individual  participant.  For  involuntary 
activities,  benefit  is  assumed  proportional  to  the  contribution  that  activity 
makes  to  an  individual's  annual  income. 

Insert  Figure  1 about  here 

The  result  of  this  analysis  is  shown  in  Figure  1.  From  this  figure 
and  related  analyses,  Starr  concludes  that  (1)  the  public  seems  willing  to 
accept  voluntary  risks  roughly  1,000  times  greater  than  involuntary  risks  at 
a given  level  of  benefit;  (2)  the  acceptability  of  a risk  is  roughly  propor- 
tional to  the  real  and  perceived  benefits;  and  (3)  the  acceptable  level  of 
risk  is  inversely  related  to  the  number  of  persons  participating  in  an 
activity.  Noting  the  similarity  between  risks  accepted  voluntarily  and 
the  risks  of  disease,  Starr  (1969,  p.  1235)  conjectures  that:  "The  rate  of 

death  from  disease  appears  to  play,  psychologically,  a yardstick  role  in 
determining  the  acceptability  of  risk  on  a voluntary  basis." 

The  Starr  approach  provides  an  intuitively  appealing  solution  to  a 
problem  facing  all  risk-benefit  analyses  and,  in  fact,  a similar  approach 
has  already  been  used  to  develop  a building  code  regulating  earthquake  risk 
in  Long  Beach,  California  (Wiggins,  1972).  There  are,  however,  a number  of 
serious  drawbacks  to  this  method.  First,  it  assumes  that  past  behavior  is 
a valid  indicator  of  present  preferences.  Second,  it  ignores  recent  psycho- 
logical research  which  has  revealed  systematic  biases  which  may  prevent  an 


individual  from  making  decisions  which  accurately  reflect  his  "true  preferences" 
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Figure  1.  Risk  (R)  plotted  relative  to  benefit  (B)  for  various  kinds  of 
voluntary  and  involuntary  exposure  (from  Starr,  1969). 
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(e.R. , Lichtenstein  & Slovic,  1971,  1973;  Slovic  & MacPhillamy , 1974).  Third, 
misperception  of  risks  as  observed  in  the  "causes  of  death"  study  described  above 
could  reduce  the  validity  of  historical  preferences  and  certainly  casts  doubt  upon 
Starr's  interpretation  regarding  the  "yardstick  role"  of  disease  rates.  Fourth,  the 
Starr  approach  assumes  that  the  public  has  available  a wide  selection  of 
alternatives  from  which  to  choose.  Is  it  reasonable  to  assume,  for  example, 
that  the  public's  automobile-buying  behavior  accurately  reflects  their  pre- 
ferences concerning  the  trade-off  between  safety  and  other  benefits?  Unless 
the  public  really  knows  what  is  possible  from  a design  standpoint,  and  unless 
the  automobile  industry  cooperates  in  making  available  information  that  may 
not  necessarily  serve  its  own  profit  maxlmation  Interests,  the  answer  is 
likely  to  be  no. 

One  avenue  of  research  that  might  help  circumvent  these  difficulties  would 
be  to  examine  risk-benefit  trade-offs  via  judgmental  techniques.  Psychological 
measures  of  perceived  risk  and  perceived  benefit  could  be  developed  for 
major  classes  of  activities.  A judgmental  space  analogous  to  Starr's  could 
then  be  constructed.  Judgments  of  desired  risk  could  be  elicited  in  addition 
to  judgments  ot  actual  risk.  Analysis  of  these  data  would  focus  on  the 
degree  to  which  judged  risk  and  benefit  correlated  with  empirical  calculations 
of  these  factors.  In  addition,  Starr's  results  regarding  voluntary  vs.  invol- 
untary activities,  level  of  perceived  benefit,  and  number  of  persons  partici- 
pating in  an  activity  could  be  validated  by  repeating  his  analyses  within 
the  judgmental  risk-benefit  space. 

Perceived  risk.  It  is  surprising  that,  with  the  exception  of  a few 
studies  using  simple  gambles  as  stimuli  (see,  for  example,  Coombs  & Huang, 

1970;  Payne,  1975),  the  determinants  of  perceived  risk  remain  unexplored. 
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j Yet,  there  is  anecdotal  and  empirical  evidence  of  a number  of  phenomena 

I 

I meriting  serious  psychological  study.  One  such  phenomenon  is  society's 

I apparent  willingness  to  spend  more  to  save  a known  life  in  danger  than 

t 

to  save  a statistical  life.  Is  this  true,  and,  if  so,  why?  Second  is 

I 

, the  speculation  that  familiarity  with  a hazard  reduces  its  perceived 

i risk.  Study  of  this  question  may  provide  Insight  into  why  the  public 

j 

I tolerates  levels  of  risk  from  some  hazards  (e.g.,  radiation  from  medical 

I x-rays)  that  they  would  never  tolerate  from  nuclear  power  plants.  A 

• third  is  the  notion  that  hazards  whose  consequences  are  delayed  (e.g.  , 

j smoking)  are  discounted.  Such  time  effects  have  rarely  been  studied. 

i 

j Fourth,  it  has  been  suggested  that  perceived  risk  is  Influenced  by  whether 

the  locus  of  uncertainty  lies  in  the  probability  that  the  hazardous 

event  will  occur  or  in  the  consequences  when  it  does  occur.  Finally, 

it  is  almost  certain  to  be  true  that  perceived  risk  depends  greatly 

upon  the  mode  of  presenting  the  relevant  information.  For  example,  risks  from 

radiation  may  appear  negligible  when  described  in  terms  of  average  reduction  in 

life  expectancy  for  the  population  within  a given  radius  of  a nuclear  power 

plant.  However,  when  this  figure  is  translaced  into  number  of  additional 

cancer  deaths  per  year,  the  risk  may  take  on  quite  a different  perspective. 

Research  on  these  phenomena  may  answer  questions  concerning  the  determi- 
nants of  societal  response  to  scientific  information  about  risk. 

Growing  concern  over  threats  to  our  environment  has  led  to  expanded 
programs  of  scientific  research  on  the  effects  of  such  hazards  as  herbicides, 
fertilizers,  pesticides,  pollution,  radiation,  etc.  It  has  been  assumed 
that  publication  of  scientific  information  about  these  hazards  would  be 
sufficient  to  elicit  appropriate  public  action.  For  reasons  which  are  inade- 
quately understood,  this  assumption  seems  not  to  be  valid.  Scientific 
Information  leads  in  some  cases  to  hasty  public  action;  in  other  cases, 


i 


29 


information  rops  unheeded  (Lawless,  1975).  While  the  determinants  of  societal 
response  are  undoubtedly  complex,  it  seems  likely  that  cognitive  factors 
related  to  communication  of  information  and  perception  of  risk  will  play 
an  important  role. 

3 life.  Although  the  economic  costs  stemming  from  prop(2rty  damage, 
disruption  of  production,  medical  expenses,  or  loss  of  earnings  can  be  esti- 
mated, we  have  no  suitable  scheme  for  evaluating  the  worth  of  a human  life 
to  society.  Despite  a certain  aversiveness  to  thinking  about  life  in  economic 
terms,  the  fact  is  Inescapable  that  by  our  actions  we  put  a finite  value  on 
our  lives.  Decisions  to  Install  safety  features,  to  buy  life  insurance, 

or  to  seek  extra  salary  for  a hazardous  job  all  carry  Implicit  values  for 
a life. 


Economists  have  debated  for  years  f-e  question  of  how  to  best  quantify  the 
value  of  a life  (see,  for  example,  HlrsMelfel,  Bergstrom  5 Rappaport , 1974; 

Hlshan,  1971;  Rloe  5 Cooper,  1967;  Schelllng,  1968),  The  traditional  economic 
approach  has  been  to  equate  the  value  of  a life  with  the  value  of  a person's  expected 
future  earnings.  Many  problems  with  this  index  are  readily  apparent.  For 
one,  it  falls  to  place  a value  on  people  in  non-income  earning  positions.  In 
addition,  it  ignores  interpersonal  effects  wherein  the  loss  suffered  by  the 


death  of  another  bears  no  relation  to  the  financial  loss  caused  by  the  death. 
A second  approach,  equating  the  value  of  life  with  court  awards  (Holmes,  1970; 


Kldner  & Richards,  1974)  is  hardly  more  satisfactory. 

Bergstrom  (1974)  argues  that  the  question  "What  is  a life  worth?"  is 
ill -formed  and  what  we  really  want  to  know  is  "What  is  the  value  placed  upon 
a specified  change  in  survival  probability?"  As  with  the  Starr  approach  to 
assessing  risk-benefit  trade-offs,  Bergstrom  argues  that  the  best  way  to 
answer  this  second  question  is  by  observing  the  actual  market  behavior  of 
people  trading  risks  for  economic  benefits.  Thus,  Thaler  and  Rosen  (1973) 
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studied  salary  as  a function  of  occupational  risk  and  found  that  a premium 
of  about  $200  per  year  «-•>«  required  to  induce  men  in  risky  occupations  to 
accept  an  annual  probability  of  .001  of  accidental  death.  From  this,  they 

inferred  that  the  value  of  life,  at  the  margin,  is  equivalent  to  about  $200,000. 
Certainly,  the  same  criticisms  leveled  earlier  at  the  Starr  approach  apply 
to  this  method.  It  assumes  that  individuals  have  enough  freedom  of  choice 
and  perceptiveness  of  risks  so  that  their  preferences  are  valid  indicators 
of  their  values. 

We  believe  this  question  is  too  Important  for  psychologists  to  Ignore. 

They  can  contribute  by  testing  the  cognitive  assumptions  upon  which  the  economic 
measures  rest  and  by  providing  alternative  methods  of  assessing  the  value  of 
a life,  such  as  direct  questions  or  other  psychophysical  techniques. 

Preliminary  attempts  at  this  by  Acton  (1973)  and  Torrance  (1970) 
have  been  downgraded  by  economists  on  the  grounds  that  "Time  and  again,  action 
has  been  found  to  contradict  assertion.  Since  surveys  always  elicit  some 
degree  of  strategic  behavior  ("What  do  they  want  me  to  say?"),  we  would  be 
better  advised  to  observe  what  people  choose  under  actual  conditions  (Rappaport, 
197A,  p.  A).  Whether  attitudes  or  behaviors  provide  a more  accurate 
reflection  of  people's  values  needs  to  be  examined  utilizing  the  broader  per- 
spective and  expertise  that  psychology  can  provide. 

Arualgamatlng  economic  and  non-economlc  costs  and  benefits.  One  of  the 
most  common  criticisms  of  risk-benefit  analysis  is  that  it  has  failed  to 
incorporate  the  many  components  of  risk  and  benefit  that  are  not  easily 
translated  into  a dollar  amount. 
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The  methodological  technique  presently  being  advocated  for  amalgamating 
economic  and  non-economlc  risks  and  benefits  Into  an  overall  Index  of 
value  suitable  for  guiding  decisions  is  "multi-attribute  utility  analysis" 

(Huber,  1974).  One  version  of  this  method  proposed  by  Edwards  (1971)  uses 
a 10-step  procedure  for  combining  multiple  outcomes  Into  an  overall  utility. 

Tnis  procedure  begins  with  structuring  the  problem  by  detailing  decision 
alternatives  and  their  attributes.  Next  the  attributes  are  ranked  in  impor- 
tance and  ^hen  relative  weights  are  assigned  to  them.  Following  this,  the 
levels  for  each  attribute  are  quantified,  judgmentally,  on  a common  scale 
of  utility.  The  next-to-last  step  Is  calculation  of  utility  for  each 
alternative  by  multiplying  scale  values  x weights  and  summing  this  over 
attributes.  The  final  step,  decision.  Involves  selecting  the  alternative 
with  the  maximum  sum.  Thus  far,  multi-attribute  procedures  such  as  this 
have  not  been  validated  and  some  would  argue  that  they  are  unvalidatable — 
a strong  statement  that  bears  scrutiny. 

1 f Ic a t ion . Decision  makers  will  employ  the  new  tools  of  risk-benefit 
analysis  and  multi-attribute  utility  theory  to  the  extent  that  they  believe 
that  such  tools  lead  to  good  decisions,  ^^at  are  the  perceived  characteristics 

of  a good  decision?  Tversky  (1972)  and  Slovlc  (1975,  In  press)  have  found  evidence 

that  decision  makers  rely  on  procedures  that  are  easy  to  explain  and  easy  to 

justify  to  themselves  and  others.  If  this  is  generally  true,  it  may  be  that 
pre-deci.sional  cognitive  processes  consist  essentially  of  searching  for  or 
constructing  a good  justification,  one  that  minimizes  lingering  doubts 
and  can  be  defended  no  matter  what  outcome  occurs.  For  people  accustomed 
to  relying  upon  such  justifications,  the  validity  of  risk-benefit  and  multi- 
attribute  utility  techniques  must  be  questioned.  The  early  steps  of 
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such  techniques,  which  involve  structuring  the  problem  and  detailing  alternatives 
and  their  attributes  may  be  useful  devices  for  helping  the  decision  maker 
think  deeply  and  in  relevant  ways  about  his  problem.  However,  we  have  serious 
reservations  about  the  quantification  steps  In  which  we  may  be  forcing  people 
to  produce  Information  at  a level  of  precision  that  does  not  exisu. 

An  alternative  conceptualization,  possibly  more  In  tune  with  people's 
natural  predilections,  would  have  decision  makers  act  like  debaters,  marshalling 
thorough  and  convincing  arguments  relevant  to  the  decision  at  hand,  rather 
than  like  computers  making  decisions  on  the  basis  of  arithmetic  (for  a similar 
argument,  see  Mason,  1969), 

These  speculations  lead  naturally  to  the  questions:  "What  are  the 

components  of  justifiability?  What  makes  a good  justification?"  Although 
we  don  t have  any  firm  answers,  we  do  have  some  hypotheses  about  factors 
that  might  noj^  be  viewed  favorably.  We  think  subjective  factors,  such  as 
subjective  trade-off  functions  or  opinions  about  probability  not  well  supported 
by  frequentlstlc  data  will  be  perceived  as  weak  justifications  for  decisions 
In  the  face  of  risk.  Subjective  probabilities  leave  one  vulnerable  to 
second  guessing — remember  the  designers  of  the  Edsel  explaining  In  1961 
that  their  carefully  constructed  opinions  about  the  market  indicated  that 
it  was  likely  to  be  a big  seller.  Expected  value  computations,  another 
basic  tool  of  the  analytic  approach  to  decision,  may  also  make  weak  justi- 
fications because  of  their  dependence  on  "long  run"  estimates;  such  estimates 
may  not  appear  relevant  for  decisions  viewed  as  one-shot  affairs. 

Will  people  view  decisions  based  on  shallow  but  nlce-soundlng, 
rationales  (cliches,  universal  truths,  adages)  as  better  than  decisions 
based  u,on  complex,  thorough  decision-analytic  techniques?  The  answer  to 
this  question  obviously  has  important  implications  for  understanding  and 
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predicting  public  decision  makers*  tcsponses  to  Information  bearing  upon 
technological  risk.  Roback  (1972,  p.  133)  In  discussing  the  defeat  of  the 
Supersonic  Transport  (SST)  subsidy,  provides  anecdotal  evidence  In  support 
of  this  conjecture. 

"There  was  not  ...  a nice  weighing  of  risk  and  benefit.  . ..  What 
counted  most  In  the  balance,  I daresay,  was  the  question  that  enough 
congressmen  put  to  themselves  before  casting  a vote:  *How  will  I 

explain  to  my  constituents,  the  majority  of  whom  have  never  even  been 
on  an  airplane  or  traveled  far  from  home,  why  we  need  an  SST  to  save 
two  or  three  hours*  travel  time  between  New  York  and  Paris?*" 

If  these  hypotheses  are  true,  the  risk-benefit  analyst  could  be  preparing 

analyses  merely  for  his  own  edification,  since  few  others  would  be  likely 

to  use  them.  In  this  event,  research  would  be  vital  to  teach  us  how  to 

communicate  risk-benefit  and  other  valuable  analytic  concepts  In  a way 

that  would  enable  such  material  to  be  woven  Into  the  fabric  of  convincing 

justifications. 

Concluding  Remarks 

Our  alms  have  been  to  summarize,  from  our  own  perspective,  the  state 
of  psychological  knowledge  regarding  decision  making  under  risk  and  to  attempt 
to  convey  our  sense  of  excitement  regarding  the  potential  contributions  of 

this  branch  of  cognitive  psychology  to  basic  knowledge  and  societal  well- 
being. 

Our  knowledge  of  the  psychological  processes  Involved  In  risk-taking 
decisions  has  Increased  greatly  in  recent  years.  However,  we  still  have 
only  a rudimentary  understanding  of  the  ways  In  which  bounded  rationality 
manifests  Itself.  We  know  much  about  certain  types  of  deficiencies  and 
biases,  but  we  don't  know  the  full  extent  of  their  generality  across  tasks 
and  across  Individuals  of  va-ylng  expertise.  Nor  do  we  know  how  to  combat 
those  biases.  We  still  do  not  understand  the  psychological  components  of 
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value  and  how  tliey  determine,  or  depend  upon,  decisions.  We  know  little 
about  perceived  risk,  the  determinants  of  societal  response  to  threat,  modes 
of  communicating  information  about  risk,  or  the  role  of  justifications  in 
decision  processes.  Research  in  these  problem  areas  is  vital  to  the  develop- 
ment of  methodologies  for  societal  decision  making  that  can  accomodate  the 
limitations  and  exploit  the  specialties  of  the  people  who  must  perform  and 


consume  these  analyses. 

H.  G.  Wells  once  commented:  "Statistical  thinking  will  one  day  be  as 

important  for  good  citizenship  as  the  ability  to  read  and  write.  That  day 
has  arrived.  Our  discussion  points  to  the  need  for  educating  both  the 
technical  experts  and  the  public  regarding  the  subtleties  of  statistical 
thinking.  Such  education  should  be  incorporated  into  the  curriculum  of  the 
schools,  perhaps  as  early  as  in  the  lower  grades.  We  need  to  teach  people 
to  recognize  explicitly  the  existence  of  uncertainty  and  how  to  deal  rationally 
with  it.  We  must  become  accustomed  to  monitoring  our  decisions  for  consistency. 
We  need  to  understand  that  the  quality  of  a decision  cannot  be  gauged  solely 
by  the  quality  of  its  outcome.  We  must  recognize  the  distortions  of  hindsight 


when  we  evaluate  the  past. 

Although  the  concept  of  bounded  rationality  arose  within  the  mainstream 
of  cognitive  psychology  (e.g.,  Miller's  and  Simon's  work),  research  on 
decision  processes  has  made  little  subsequent  contact  even  with  such  closely 
related  fields  as  the  study  of  non-probabilistic  information  processing.  It 
sliould.  Certainly  the  phenomena  described  here  cannot  be  fully  understood 
without  considerations  of  their  underlying  cognitive  mechanisms.  Likewise, 
some  of  these  phenomena  may  provide  stimulating  inputs  for  general  theories 
of  cognition.  The  hindsight  results,  for  example,  indicate  one  way  in  which 
semantic  memory  is  reorganized  to  accommodate  new  information.  The  bias 
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here  called  availability  suggests  a need  to  better  understand  the  process 
of  constrained  associates  production.  No  theory  of  cognitive  development 
appears  to  relate  to  the  acquisition  of  judgmental  biases  and  heuristics  as 
conceptualized  here.  VJlthout  such  knowledge,  we  have  no  idea  when  it  is 
best,  or  when  it  is  even  possible,  to  begin  teaching  children  to  think 
probabilistically. 

While  this  article  has  emphasized  what  psychologists  can  do  to  faci- 
litate .societal  decision  making,  clearly  a multidisciplinary  approach, 
involving  cooperative  efforts  with  physicists,  economists,  engineers,  geogra- 
phers. and,  perhaps  most  Important,  decision  makers,  is  called  for.  Only 
by  working  hand  in  hand  with  decision  makers  can  we  learn  what  their  problems 
are--both  those  they  perceive  and  those  they  do  not.  Only  continual  multi- 
disciplinary interaction  will  alert  us  to  uue  narrowness  of  our  own  perspec- 
tive and  enable  us  to  develop  practical  tools  for  decision  makers. 
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